Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Redesign PatchWSIDataset #4152

Merged
merged 17 commits into from
Apr 22, 2022
Merged

Redesign PatchWSIDataset #4152

merged 17 commits into from
Apr 22, 2022

Conversation

drbeh
Copy link
Member

@drbeh drbeh commented Apr 20, 2022 •
edited
Loading

Fixes #3930

Description

This PR refactor PatchWSIDataset and uses new WSIReader as the backend.

Status

Ready

Types of changes

  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@drbeh
Implement PatchWSIDataset
6ac6545 
Signed-off-by: Behrooz <[email protected]>
@drbeh
Add unittests
d227b55 
Signed-off-by: Behrooz <[email protected]>
@drbeh
Add docs
a2d9a11 
Signed-off-by: Behrooz <[email protected]>
@drbeh
Reorder imports
f1324d1 
Signed-off-by: Behrooz <[email protected]>
@drbeh
formatting:
39a679a 
Signed-off-by: Behrooz <[email protected]>
@drbeh drbeh requested review from Nic-Ma and wyli April 20, 2022 18:37
Nic-Ma
Nic-Ma reviewed Apr 21, 2022
View reviewed changes
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 21, 2022 •
edited
Loading

Hi @drbeh ,

BTW, about the SmartCachePatchWSIDataset, I think your previous suggestion was correct, we can slightly optimize the CacheDataset and SmartCacheDataset, etc. Then we can use them as decorators on another dataset object in future.

For now, you can use PatchWSIDataset as the data arg of SmartCacheDataset in your program to replace:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/pathology/data/datasets.py#L166

We must implement SmartCachePatchWSIDataset before because Clara 4.0 only supports 1 dataset in the config, now it's not necessary anymore.

Thanks.

@drbeh
Address comments
688ff04 
Signed-off-by: Behrooz <[email protected]>
@drbeh
Copy link
Member Author

drbeh commented Apr 21, 2022

Hi @drbeh ,

BTW, about the SmartCachePatchWSIDataset, I think your previous suggestion was correct, we can slightly optimize the CacheDataset and SmartCacheDataset, etc. Then we can use them as decorators on another dataset object in future.

For now, you can use PatchWSIDataset as the data arg of SmartCacheDataset in your program to replace: https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/pathology/data/datasets.py#L166

We must implement SmartCachePatchWSIDataset before because Clara 4.0 only supports 1 dataset in the config.

Thanks.

Sure, will add SmartCache in a seperate PR as the test cases need some attention.

@drbeh
Copy link
Member Author

drbeh commented Apr 22, 2022

Hi @Nic-Ma, any other comment here? thanks :)

@drbeh
Add new test cases
bab87be 
Signed-off-by: Behrooz <[email protected]>
@drbeh
sorts
c725103 
Signed-off-by: Behrooz <[email protected]>
Nic-Ma
Nic-Ma approved these changes Apr 22, 2022
View reviewed changes
Copy link
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update.
Looks good to me.

monai/data/wsi_datasets.py Show resolved Hide resolved
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 22, 2022

/black

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 22, 2022

/build

@drbeh drbeh enabled auto-merge (squash) April 22, 2022 13:56
@drbeh drbeh merged commit 65d7296 into Project-MONAI:dev Apr 22, 2022
@drbeh drbeh mentioned this pull request May 2, 2022
Closed
@drbeh drbeh deleted the new-wsi-datasets branch May 2, 2022 15:39
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request May 10, 2022
@drbeh @Can-Zhao
Redesign PatchWSIDataset (Project-MONAI#4152)
e56e75d 
* Implement PatchWSIDataset

Signed-off-by: Behrooz <[email protected]>

* Add unittests

Signed-off-by: Behrooz <[email protected]>

* Add docs

Signed-off-by: Behrooz <[email protected]>

* Reorder imports

Signed-off-by: Behrooz <[email protected]>

* formatting:

Signed-off-by: Behrooz <[email protected]>

* Address comments

Signed-off-by: Behrooz <[email protected]>

* Update to be compatible with Dataset

Signed-off-by: Behrooz <[email protected]>

* Update reader to accept str, class, object

Signed-off-by: Behrooz <[email protected]>

* Add test cases for various reader and level arguments

Signed-off-by: Behrooz <[email protected]>

* Update comment about OpenSlide cache

Signed-off-by: Behrooz <[email protected]>

* Rename reader_name to backend

Signed-off-by: Behrooz <[email protected]>

* Add new test cases

Signed-off-by: Behrooz <[email protected]>

* Add unittests for openslide

Signed-off-by: Behrooz <[email protected]>

* Add new test cases

Signed-off-by: Behrooz <[email protected]>

* sorts

Signed-off-by: Behrooz <[email protected]>

* Add docstring for kwargs

Signed-off-by: Behrooz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Nic-Ma Nic-Ma Nic-Ma approved these changes

@wyli wyli Awaiting requested review from wyli

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Refactor PatchWSIDataset
2 participants
@drbeh @Nic-Ma